feat(parser): add Shelley (exe.dev) agent support - #718
Conversation
Shelley stores every conversation in a single SQLite DB at ~/.config/shelley/shelley.db (conversations + messages tables). It is a single-DB SQLite agent structurally identical to Zed, so it mirrors the Zed sync wiring: a virtual source path (shelley.db#<conversationID>), classify-before-pathExists, a WAL/SHM composite mtime, and per-session forceReplace on parse. messages.llm_data is a serialized llm.Message with PascalCase keys and integer Role/Content-type enums (the one parsing gotcha); usage_data already uses the canonical Anthropic token keys, so the raw blob is stored verbatim for cost pricing. All generations are included ordered by sequence_id, which is monotonic per conversation, so a context-reset (distillation) boundary never hides earlier history.
- SourceMtime: add a Shelley branch so virtual shelley.db#<id> paths resolve to the conversation updated_at instead of falling through to os.Stat (which returns 0 and makes the live per-session watcher treat the source as gone). Adds parser.ShelleySourceMtime. - Capture token usage on errored assistant turns (stored as type="error"), not just type="agent"; applyShelleyUsage now no-ops on the all-zero usage blobs Shelley writes for user/tool rows. - Note that exact gateway cost_usd is available but deferred (capturing it without double-counting catalog-priced tokens needs a usage-event path); standard models price correctly via the catalog today. - Tests: SourceMtime virtual-path resolution, real space-separated DATETIME format, and robustness (unknown content type, redacted thinking, malformed llm_data, errored-turn token capture). - Frontend: distinct color/label for Shelley sessions.
Verified against a real shelley.db: content Type integers are text=2, thinking=3, tool_use=5, tool_result=6 (not 0/1/3/4). In the upstream llm package, ContentType shares a const block with MessageRole (User=0, Assistant=1) and iota does not reset between the two groups, so the content types start at 2. The unit fixtures previously used the same wrong values as the parser, so they passed while disagreeing with reality; they are now corrected too.
roborev: Combined Review (
|
Shelley web_search_tool_result blocks nest their results as web_search_result blocks (content type 9) that carry Title/URL instead of Text. shelleyToolResultText only extracted Text, so these results flattened to an empty string and the carrier message could be dropped. Handle Title/URL blocks in shelleyToolResultText, join flattened blocks with newlines so multiple results stay readable, and add a test covering a nested web_search_tool_result.
|
Fixed in cd9223f.
Context worth flagging: I caught and cleared this locally before pushing, because as of this session I finally have a matching roborev configuration — a two-reviewer codex (GPT-5.5) panel (correctness + security) mirroring |
roborev: Combined Review (
|
Shelley's updated_at is SQLite CURRENT_TIMESTAMP (whole-second precision), so keying per-conversation skip detection on it alone could skip a conversation rewritten within the same wall-clock second even though its content changed. The mirrored Zed parser is unaffected: Zed writes sub-second RFC3339 timestamps, so its skip signal is already collision-free. Fold Shelley's own monotonic max(sequence_id) cursor -- the signal that powers Shelley's incremental-fetch protocol -- into the File.Mtime change signal as updatedAtNanos + maxSeq. All three signal sites (stored File.Mtime, the meta skip query, and ShelleySourceMtime) share the shelleyChangeMtime helper so they stay consistent. No engine, schema, or Zed change is needed and File.Mtime drift stays sub-millisecond.
roborev: Combined Review (
|
Fold a per-conversation content byte-length sum into the Shelley change signal so the sync skip detects an in-place message rewrite that keeps updated_at in the same wall-clock second and appends no new row (sequence_id unchanged). The meta query computes the sum in SQLite via LENGTH(CAST(... AS BLOB)); the parse loop sums the same bytes it already reads, so both sides match exactly. Add ParseShelleyVirtualPath to stripVirtualSourceSuffix so stored shelley.db#id rows map back to the real shelley.db. Without it, a DB read failure or dropped session for a stored Shelley conversation was reported as source-missing instead of parse drift.
roborev: Combined Review (
|
Fold the Shelley change signal's max sequence_id and content-byte-length into a sub-second offset via an FNV-1a hash rather than adding them to the timestamp. A plain additive sum let the two cancel (a +1 sequence_id paired with a -1 byte delta produced an unchanged signal), so the sync skip could still miss a same-second append-plus-shrink. Folding the hash into [0, 1s) also keeps file_mtime within the conversation's reported second, so it stays a valid nanosecond timestamp for the range queries in ListSessionsModifiedBetween.
Replace the additive/byte-length Shelley change signal with two per-conversation signals so the sync skip cannot miss a rewrite: - file_mtime stays the conversation's real updated_at timestamp. It must remain a true timestamp because ListSessionsModifiedBetween filters file_mtime <= now for PG/DuckDB push; a synthetic future value dropped a just-synced Shelley row from a same-second push until a later run. - file_hash holds a content digest over the conversation's messages, computed identically by the meta skip query and the parse loop. It is length-framed, so it detects appends, in-place rewrites, and length-preserving same-second edits that a byte-length signal misses. The bulk skip now compares the stored file_hash via GetFileHashByPath alongside file_mtime. Computing the digest reads message payloads, which the sub-second-timestamped siblings (Zed, Kiro) avoid; Shelley's second-precision updated_at makes the read the price of not silently skipping a rewrite. ShelleySourceMtime keeps a watcher-only change signal (updated_at plus a sub-second digest term), compared for inequality and never range-filtered.
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
2 participants
Adds first-class support for Shelley, the exe.dev / boldsoftware coding agent (github.com/boldsoftware/shelley), so its sessions appear alongside every other agent.
Storage model
Shelley stores all conversations in a single SQLite DB at
~/.config/shelley/shelley.db(conversations+messagestables). This is structurally identical to Zed, so the parser and sync wiring mirror the Zed single-DB pattern: a virtual source path (shelley.db#<conversationID>), classification before thepathExistsguard, a WAL/SHM composite mtime, per-sessionforceReplaceon parse, and aSourceMtimebranch for the live per-session watcher. NoResyncAlloldFileSessionsaccounting is needed (that path is specific to the OpenCode-format storage agents) — Shelley re-parses fresh from the present DB, and vanished conversations are preserved via the generic orphan-copy.Extraction
messages.llm_datais a serializedllm.Messagewith PascalCase keys and integer enums. The contentTypeintegers start at 2 (text=2, thinking=3, tool_use=5, tool_result=6) becauseContentTypeshares aniotaconst block withMessageRoleandiotadoes not reset between the two groups; this was verified against a realshelley.db. Tool results are stored asuser-role messages (Anthropic-style) and pair into the originating tool call. All generations are included, ordered bysequence_id(monotonic and unique per conversation), so a context-reset / compaction boundary never hides earlier history.Tokens & cost
usage_dataalready uses the canonical Anthropic token keys, so the raw blob is stored verbatim and cost is catalog-priced. Token usage is also captured on errored assistant turns (stored astype="error"). The exact gatewaycost_usdis present in the payload but is currently 0 from the exe.dev gateway; capturing it without double-counting the catalog-priced per-message tokens is left as a follow-up.Where to look
internal/parser/shelley.go— parser, virtual-path / read-only store helpers, content and token extraction.internal/sync/engine.go— the Shelley sites mirroring Zed (classify, composite mtime,processShelley, dispatch,SourceMtime, cache-skip).internal/parser/types.go— theAgentShelleyregistry entry.Limitations
cost_usdcapture is deferred (see above); standard gateway models are priced correctly by the catalog.shelley.dbdata; tool-name categories cover the current Shelley tool set (bash,patch,keyword_search,browser*,subagent, …).